Comparison of Entropy and Mean Square Error Criteria in Adaptive System Training Using Higher Order Statistics
نویسندگان
چکیده
The error-entropy-minimization approach in adaptive system training is addressed in this paper. The effect of Parzen windowing on the location of the global minimum of entropy has been investigated. An analytical proof that shows the global minimum of the entropy is a local minimum, possibly the global minimum, of the nonparametrically estimated entropy using Parzen windowing with Gaussian kernels. The performances of error-entropy-minimization and the mean-square-errorminimization criteria are compared in short-term prediction of a chaotic time series. Statistical behavior of the estimation errors and the higher order central moments of the time series data and its predictions are utilized as the comparison criteria.
منابع مشابه
Convergence properties and data efficiency of the minimum error entropy criterion in ADALINE training
Recently, we have proposed the minimum error entropy (MEE) criterion as an information theoretic alternative to the widely used mean square error criterion in supervised adaptive system training. For this purpose, we have formulated a nonparametric estimator for Renyi’s entropy that employs Parzen windowing. Mathematical investigation of the proposed entropy estimator revealed interesting insig...
متن کاملMinimizing the Fisher Information of the Error in Supervised Adative Filter Training
In this paper, we propose minimizing the Fisher information of the error in supervised training of linear and nonlinear adaptive filters. Fisher information considers the local structure of the error probability distribution and therefore, it is expected to result in more robust solutions compared to other statistics such as minimum mean-square-error or minimum-error-entropy. A gradient-based t...
متن کاملAn error-entropy minimization algorithm for supervised training of nonlinear adaptive systems
This paper investigates error-entropy-minimization in adaptive systems training. We prove the equivalence between minimization of error’s Renyi entropy of order and minimization of a Csiszar distance measure between the densities of desired and system outputs. A nonparametric estimator for Renyi’s entropy is presented, and it is shown that the global minimum of this estimator is the same as the...
متن کاملMinimum Error Entropy Algorithms with Sparsity Penalty Constraints
Recently, sparse adaptive learning algorithms have been developed to exploit system sparsity as well as to mitigate various noise disturbances in many applications. In particular, in sparse channel estimation, the parameter vector with sparsity characteristic can be well estimated from noisy measurements through a sparse adaptive filter. In previous studies, most works use the mean square error...
متن کاملA Family of Variable Step-Size Normalized Subband Adaptive Filter Algorithms Using Statistics of System Impulse Response
This paper presents a new variable step-size normalized subband adaptive filter (VSS-NSAF) algorithm. The proposed algorithm uses the prior knowledge of the system impulse response statistics and the optimal step-size vector is obtained by minimizing the mean-square deviation(MSD). In comparison with NSAF, the VSS-NSAF algorithm has faster convergence speed and lower MSD. To reduce the computa...
متن کامل